Apparatus for performing loop-filtering on four successive pixels of an image

ABSTRACT

In one embodiment, the apparatus includes a decoder configured to obtain quantization information of a first block, obtain a block type of the first block, and compare the quantization information and a threshold value. Loop-filtering is performed by the decoder on four successive pixels by using at least three successive pixels if the quantization information is greater than the threshold value and the type of the first block is an intra-coded block type. Two pixels of the four successive pixels are in the first block and two pixels of the four successive pixels are in a second block adjacent to the first block.

DOMESTIC PRIORITY INFORMATION

This is a divisional of U.S. application Ser. No. 11/097,394 filed Apr.4, 2005, now U.S. Pat. No. 7,711,054, which is a divisional of U.S.application Ser. No. 10/170,999 filed Jun. 14, 2002, now issued as U.S.Pat. No. 7,272,186; the contents of all of which are hereby incorporatedby reference in their entirety.

FOREIGN PRIORITY INFORMATION

The present invention claims priority under 35 U.S.C. 119 on KoreanApplication No. 2001-33953 filed Jun. 15, 2001; the contents of whichare hereby incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to a moving picture compression technique,and more particularly, to a loop filtering method in a video coder.

2. Description of the Background Art

Generally, it has been proved that the “H.26L” system developed as anext generation moving picture compression technique is superior to the“H.263” and “MPEG4” systems in performance.

The “H.26L” system as the next generation moving picture compressionsystem differs from the conventional H.263 and MPEG4 in using 4×4 blockbased transformation & encoding, carrying out motion estimation &compensation of a transformation block size, and using a single variablelength coder (VLC).

Superior to the conventional motion picture standards in aspect ofperformance, the H.26 L system has excessive calculation of encoder aswell as uses 4×4 block based transformation so that a blocking dominatesthan the conventional standards.

In order to remove the blocking, filters include a loop filtering systeminside the encoder and decoder, a post filter processed in a next stageof the decoder, and a system adding the loop filtering system to thepost filter.

When the blocking is removed by the loop filtering system in general, adifference between an input video and a motion video is affected so thatan encoding rate increases in a specific case. Yet, as the loopfiltering system is installed inside a video coder, complexity of theencoder increases.

As mentioned in the above explanation, the H.26 L system according tothe related art requires excessive quantity of calculation despite thesuperiority in performance to the moving picture standards as well ashaving the blocking more dominant than that of the moving picturestandards due to the 4×4 block based transformation system.

Moreover, if the blocking is removed by the loop filtering systemaccording to the related art, the difference between the input video andthe motion video is affected so as to increase the encoding rate in aspecific case. Yet, the loop filtering system built inside the videocoder increases the complexity of the encoder.

SUMMARY OF THE INVENTION

The present invention relates to an apparatus for performingloop-filtering on pixels of an image.

In one embodiment, the apparatus includes a decoder configured to obtainquantization information of a first block, obtain a block type of thefirst block, and compare the quantization information and a thresholdvalue. Loop-filtering is performed by the decoder on four successivepixels by using at least three successive pixels if the quantizationinformation is greater than the threshold value and the type of thefirst block is an intra-coded block type. Two pixels of the foursuccessive pixels are in the first block and two pixels of the foursuccessive pixels are in a second block adjacent to the first block.

It is to be understood that both the foregoing general description andthe following detailed description of the present invention areexemplary and explanatory and are intended to provide furtherexplanation of the invention as claimed.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings, which are included to provide a furtherunderstanding of the invention and are incorporated in and constitute apart of this application, illustrate embodiment(s) of the invention andtogether with the description serve to explain the principle of theinvention. In the drawings:

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention;

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively;

FIG. 3 illustrates a diagram of pixel locations of adjacent blocksaccording to an embodiment of the present invention; and

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention.

DETAILED DESCRIPTION OF THE EXAMPLE EMBODIMENTS

Reference will now be made in detail to the example embodiments of thepresent invention, which are illustrated in the accompanying drawings.

Hereinafter, a loop filtering method in a video coder according to anexample embodiment of the present invention carries out a real-timeprocessing of removing a blocking and a ringing phenomenon in accordancewith a pixel location, a coded block pattern, a quantization step size,and a motion vector size in a digital video appliance using the H.26Lmoving picture compression technique, thereby enabling to improve avideo quality of a compression image demanding a low bit rate orhigh-speed processing.

First, explained in detail is a real-time processing technique ofremoving “blocking” and “ringing” by predicting an original video forcorrelation between pixels in accordance with a quantization variableQP, a coded block pattern, and a motion vector size available for adecoder (not shown in the drawing) in a digital video appliance.

When an original video f is transmitted as a compressed form, a videorestructured in a decoder can be expressed by the following Formula 1-1.g=f+n,  [Formula 1-1]where g, f, and n are a compression video rearranged in a scanningorder, an original video, and a column vector of quantization error,respectively.

A relation between respective pixels can be expressed by the followingFormula 1-2 in order to process the Formula 1-1 by pixel unit.g(i,j)=f(i,j)+n(i,j),  [Formula 1-2]where i and j indicate a location of a pixel in vertical and horizontaldirections.

An additional function can be defined as the following Formula 1-3 whenreliability of original pixels and 1-dimensional smoothing function foreach pixel are used together with a regularization. Firstly, ahorizontal direction is considered.M{f(i,j)}=M _(p) {f(i,j)}+M _(N) {f(i,j)},  [Formula 1-3]where M_(p) and M_(N) are functions representing reliability smoothingdegrees of left and right directions for a pixel f(i,j), respectively.

And, such functions are defined as the following Formula 1-4, where theM_(P) and M_(N) are the functions considering f(i,j) & f(i,j−1) andf(i,j) & f(i,j+1), respectively.M _(P) {f(i,j)}=α_(P) {f(i,j)−f(i,j−1)}²+(1−α_(P)){g(i,j)−f(i,j)}²M _(N) {f(i,j)}=α_(N){f(i,j)−f(i,j−1)}²+(1−α_(N)){g(i,j)−f(i,j)}²  [Formula 1-4]

Each of the first terms of right sides in Formula 1-4 indicatesirregularity with an adjacent pixel, i.e. a difference from an adjacentpixel, for the pixel f(i,j), and each of the second terms indicates areliability for the original pixel f(i,j). Moreover, α_(P) and α_(N)mean regularization parameters representing ratios between irregular andreliability terms defined in the functions, respectively. And, α_(P) andα_(N) are parameters adjusting a relative coherence between the twoterms in Formula 1-4.

With the above manner, the additional function is defined for each pixelof the moving picture, and the function in a vertical direction isdefined by varying the parameter i instead of the parameter j in Formula1-4.

Thereafter, if a differential is applied to the additional functions inFormula 1-4 defined for each pixel of the moving picture in order tofind a restored video from which the blocking and ringing are removed,the following Formulas 1-5 and 1-6 are attained.

$\begin{matrix}{\frac{{\partial M}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} + \frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}}} = 0}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}5} \right\rbrack\end{matrix}$

From Formula 1-5, the following Formula 1-6 is found.

$\begin{matrix}{{\frac{{\partial M_{P}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{P}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{P}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}{\frac{{\partial M_{N}}\left\{ {f\left( {i,j} \right)} \right\}}{\partial{f\left( {i,j} \right)}} = {{2{\partial_{N}\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}}} - {2\left( {1 - \alpha_{N}} \right)\left\{ {{g\left( {i,j} \right)} - {f\left( {i,j} \right)}} \right\}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}6} \right\rbrack\end{matrix}$

From Formulas 1-5 and 1-6, the pixel to be restored in a horizontaldirection can be found by the following Formula 1-7.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{f\left( {i,{j - 1}} \right)}} +} \\{\alpha_{N}{f\left( {i,{j + 1}} \right)}}\end{matrix}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}7} \right\rbrack\end{matrix}$

As the regularization parameters in Formula 1-7 are between “0” and “1”,Formula 1-7 can be defined by the following Formula 1-8.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2 - \alpha_{P} - \alpha_{N}} \right){g\left( {i,j} \right)}} + {\alpha_{P}{g\left( {i,{j - 1}} \right)}} +} \\{\alpha_{N}{g\left( {i,{j + 1}} \right)}}\end{matrix}}{2}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}8} \right\rbrack\end{matrix}$

Looking into Formula 1-8, a blocking-removed video of the compressionvideo is determined by the two, left and right pixel values and theregularization parameters by taking the i_(th) and j_(th) videos asreferences.

As the two, left and right pixel values can be used for encoding anddecoding, two regularization parameter values are set up for attainingthe restored video. For this, “set theoretic” is applied to therestoration by pixel unit so as to represent regularization coefficientsby the following Formula 1-9, where “set theoretic” is a theory forcontrolling the parameters α_(P) and α_(N) in Formula 1-4.

$\begin{matrix}{{\frac{\alpha_{P}}{1 - \alpha_{P}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j - 1}} \right)}} \right\}^{2}} \approx \frac{K_{P}{QP}^{2}}{\left\{ {{q\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right)}} \right\}^{2}}}},{\frac{\alpha_{N}}{1 - \alpha_{N}} = {\frac{\left\{ {{f\left( {i,j} \right)} - {g\left( {i,j} \right)}} \right\}^{2}}{\left\{ {{f\left( {i,j} \right)} - {f\left( {i,{j + 1}} \right)}} \right\}^{2}} \approx \frac{K_{N}{QP}^{2}}{\left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j + 1}} \right)}} \right\}^{2}}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}9} \right\rbrack\end{matrix}$

K_(P) and K_(N) in Formula 1-9 are determined statistically as adaptiveparameters depending on a location of a pixel since the irregularitiesbetween pixels at a block boundary and an inner block side are differentfrom each other. And, “QP” is a quantization parameter of a macro blockto which the pixel f(i,j) belongs.

Even though the restored video from which blocking is removed usingFormulas 1-8 and 1-9, the value of the regularization parameter requiresa floating-point operation so as to become a problem in aspect ofcalculation quantity of the filter to remove the “blocking”.

Therefore, Formulas 1-8 and 1-9 are transformed into the followingFormulas 1-10 and 1-11 for integer operation.

$\begin{matrix}{{f\left( {i,j} \right)} = \frac{\begin{matrix}{{\left( {2^{9} - {\alpha_{P} \times 2^{8}} - {\alpha_{N} \times 2^{8}}} \right){g\left( {i,j} \right)}} +} \\{{\alpha_{P}{2^{8} \times {g\left( {i,{j - 1}} \right)}}} + {\alpha_{N} \times 2^{8} \times {g\left( {i,{j + 1}} \right)}}}\end{matrix}}{{2 \times 2^{8}}\;}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}10} \right\rbrack \\{{\beta_{P} = {{\alpha_{P} \times 2^{8}} = \frac{{2^{8} \times K_{P}}{QP}^{2}}{{K_{P}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}},{\beta_{N} = {{\alpha_{N} \times 2^{8}} = \frac{{2^{8} \times K_{N}}{QP}^{2}}{{K_{N}{QP}^{2}} + \left\{ {{g\left( {i,j} \right)} - {g\left( {i,{j - 1}} \right\}}^{2}} \right.}}}} & \left\lbrack {{Formula}\mspace{14mu} 1\text{-}11} \right\rbrack\end{matrix}$

In this case, β_(P) and β_(N), as defined in Formula 1-11, are variablesattained by multiplying α_(P) and α_(N) by 2⁸, respectively. Namely,β_(P) and β_(N) are parameters for making α_(P) and α_(N) into integers.

In an algorithm according to the present invention, the regularizationparameters as integer forms using Formula 1-11 are generated, and thenstored as a look-up table form to be used.

Hence, Formula 1-10 for blocking filtering can be found by knowing thepixel f(i,j) and locations and quantization parameter values of twopixels adjacent to the pixel f(i,j).

Meanwhile, in the loop filtering process of the present invention, avideo of which blocking in a direction of a horizontal axis is removedby loop-filtering a block of a moving picture is attained, and then thevideo is loop-filtered in a direction of a vertical axis so as toprovide the video of which blocking in a direction of a vertical axis isremoved.

FIG. 1 illustrates a flowchart of a loop filtering process according toan embodiment of the present invention.

Referring to FIG. 1, a loop filtering method according to an embodimentof the present invention includes the steps of determining a strength ofa block to be processed in accordance with a coded pattern and aquantization step size of each 4×4 block when a block of a movingpicture is loop-filtered in directions of its horizontal and verticalaxes (S11), carrying out a loop filtering in accordance with thedetermined strength of the corresponding block (S12), and carrying outan additional loop filtering at a boundary region of a macro block sincea relatively big blocking exists at the boundary region of the macroblock (S13). This is explained in detail by referring to FIG. 2A, FIG.2B, and FIG. 3.

FIGS. 2A and 2B illustrate block diagrams indicating block locations inhorizontal and vertical directions according to an embodiment of thepresent invention, respectively, and FIG. 3 illustrates a diagram ofpixel locations of adjacent blocks according to an embodiment of thepresent invention, in which block locations in directions of horizontaland vertical axes are shown.

First of all, the step of determining the block strength St and afiltering state is explained as follows.

The step of determining the block strength St and a filtering state is aprocess for determining the H.26L system as standards, for which TML(test model long-term number 5) is in progress. In this case, the TMLcompresses a video by taking a 4×4 block as a reference.

The TML (test model long-term) moving picture compression system uses atechnique of removing spatial and temporal redundancy information froman encoder so as to transmit spatially & temporally compressedinformation and additional information required for decoding. Of course,in order to remove the redundancy information on a temporal domain, MPEG(moving picture expert group) and H.263 motion compensation systemaccording to the related art are used. The technique of removing theblocking using a basic mechanism of such a TML system is generallytreated by 4×4 block unit.

In this case, H.263 and H.26 L are moving picture compression standardsproposed by ITU-T (international telecommunicationsunion-telecommunication).

The system for removing the blocking according to an embodiment of thepresent invention may follow the system of the related art. Yet, thefiltering state is determined by the following pre-treatment steps 1 to3 since the degree of the blocking differs in accordance with a codedblock pattern, a quantization step size, and a motion vector size.

[Step 1]

Strength St of each 4×4 block is initialized by ‘0’ (St=0).

[Step 2]

If each of the 4×4 blocks is an intra-coded block or has a non-zerotransform coefficient, St=max(St,2).

[Step 3]

If a motion vector difference between respective motion vectors for thetwo blocks block1 and block2 in FIG. 2( a) and FIG. 2( b) is equal to orgreater than 1′, St=Max(St, 1).

In this case, max(a,b) means a maximum value of ‘a’ and ‘b’. Namely,max(a,b) is a function for selecting a greater value from ‘a’ and ‘b’.

Moreover, after the strength St of each of the blocks has beendetermined, the loop filtering is carried out in accordance with thequantization size of each of the blocks.

For carrying out the loop filtering, the quantization sizes of the twoblocks block1 and block2 are as follows.

First case is that the quantization value of the block block1 is smallerthan ‘21’ and the strengths of the two blocks block1 and block2 are not‘0’. And, second case is that the quantization value of the block block1is greater than ‘20’ and the strength St of one of the two blocks block1and block2 is not ‘0’.

In this case, the filtering state is determined differently inaccordance with the quantization size because an adaptive treatment isperformed due to the dominating generation of blocking in case of agreat quantization size.

Therefore, when the conditions for the pre-treatment steps 1 to 3 aremet, the loop filtering process is carried out by the following manner.

First, pixel locations of two blocks block1 and block2 are shown in FIG.3. If the above-defined filtering state of each of the blocks issatisfied, values of pixels c, d, e, and f are found by the followingFormula 2-1 (for calculating filter coefficient values to be used inFormula 2-5).r1=reg{St ₁ ,QP,abs(b−c)},r2=reg{St ₁ ,QP,abs(c−d)},r3=reg{max(St ₁+1,St ₂+1),QP,abs(d−e)},r4=reg{St ₂ ,QP,abs(e−f)},r5=reg{St ₂ ,QP,abs(f−g)}  [Formula 2-1]

From Formula 2-1, reg(.) is defined by the following Formula 2-2.

$\begin{matrix}{{{{reg}\left( {{St},{QP},{Diff}} \right)} = \frac{256 \times {M({St})} \times {{DQP}({QP})}}{{{Diff} \times {Diff}} + {{M({St})} \times {{DQP}({QP})}}}},} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}2} \right\rbrack\end{matrix}$where Diff is a random value and ‘abs’ indicates a function of absolutevalue.

And, M(St) is defined by the following Formula 2-3.M(St=0)=⅙,M(St=1)= 4/16,M(St=2)= 8/16,M(St=3)=1  {Formula 2-3}

And, DQP(QP) is explained by referring to FIG. 4 as follows.

FIG. 4 illustrates a table of LQP values for determining DQP(QP) valuesaccording to an embodiment of the present invention. Namely, DQP(QP) isdefined by the following Formula 2-4, and can be found by substitutingthe LQP values shown in the table of FIG. 4. LQP is a quantization tabledefined in H.26L, and DQP defines a quantization error range inaccordance with the quantization table defined in H.26L as defined byFormula 2-4. Namely, the LQP values are “620”, “553”, “429”, . . . ,“24”, and “22” if the quantization variables QP are “0”, “1”, “2”, . . ., “30”, and “31”, respectively. Namely, as the quantization variable (0,1, 2, . . . , 30, 31) increases sequentially, the LQP value decreaseswith a predetermined value (620, 553, 492, . . . , 24, and 22).

$\begin{matrix}{{{DQP}({QP})} = \frac{676}{{LQP}({QP})}} & \left\lbrack {{Formula}\mspace{14mu} 2\text{-}4} \right\rbrack\end{matrix}$

Formula 1-11 is equivalent to Formula 2-2, and QP², K_(P)(or K_(N)), and{g(i,j)−g(i,j−1)} in Formula 1-11 are parameters equivalent to DQP(QP),M(St), and Diff in Formula 2-2, respectively.

Hence, the filtered values of the pixels c, d, e, and f are calculatedby the following Formula 2-5 using the defined parameters.c _(mod)={(512−r ₁ −r ₂)×c+r ₁ ×b+r ₂ ×d+256}>>9d _(mod)={(512−r ₂ −r ₃)×d+r ₂ ×c+r ₃ ×e+256}>>9e _(mod)={(512−r ₃ −r ₄)×c+r ₃ ×e+r ₄ ×f+256}>>9f _(mod)={(512−r ₄ −r ₅)×f+r ₄ ×e+r ₅ ×g+256}>>9,  [Formula 2-5]where “>>” means a bit shift, and C_(mod), d_(mod), e_(mod), and f_(mod)mean values of filtering c˜f having the blocking shown in FIG. 3.

Meanwhile, the result from the above-explained processing brings aboutan effect of removing the blocking and ringing with satisfaction. Yet,micro blocking still exists in the boundary region of the macro blockconstituted by 16×16 block unit. In order to settle the micro blocking,an additional filtering, as shown in Formula 2-6, is carried out on thepixel lying at the boundary region of the macro block.d _(mod)=(c _(mod)+13×d _(mod)+2×e _(mod)+8)>>4e _(mod)=(2×d _(mod)+13×e _(mod) +f _(mod)+8)>>4,if QP>16 and d and e are macro block boundary pixels.

Accordingly, the present invention carries out a real-time processing ofremoving a blocking and a ringing phenomenon in accordance with a pixellocation, a coded block pattern, a quantization step size, and a motionvector size in a digital video appliance using the H.26L moving picturecompression technique, thereby enabling to improve a video quality of acompression image demanding a low bit rate or high-speed processing.

The foregoing embodiments are merely exemplary and are not to beconstrued as limiting the present invention. The present teachings canbe readily applied to other types of apparatuses. The description of thepresent invention is intended to be illustrative, and not to limit thescope of the claims. Many alternatives, modifications, and variationswill be apparent to those skilled in the art.

1. An apparatus for removing a blocking phenomenon using properties oftwo blocks, comprising: a decoder configured to determine whether atleast one of a first block and a second block has non-zero transformcoefficient information, and the second block being adjacent to thefirst block, the decoder further configured to, determine a filterstrength value in consideration of a presence of the non-zero transformcoefficient information, obtain a pixel value difference between atleast two pixels on a block boundary between the first block and thesecond block, determine coefficients, for pixels on the block boundarybetween the first and second blocks, based on the filter strength value,and calculating values of the pixels on the block boundary based on thedetermined coefficients and the obtained pixel value difference, and abit shift operation, wherein the bit shift operation uses an integer,wherein the integer is greater than one, and wherein the bit shiftoperation is performed after using the obtained pixel value difference,and the calculating step further including, calculating values of atleast two successive pixels in the first block based on pixel values ofthe at least two successive pixels in the first block, and calculatingvalues of at least two successive pixels in the second block based onpixel values of the at least two successive pixels in the second block.2. The apparatus of claim 1, wherein the coefficients are determined tobe varied depending on the filter strength value.
 3. An apparatus forremoving a blocking phenomenon using properties of two blocks,comprising: a decoder configured to determine whether at least one of afirst block and a second block has non-zero transform coefficientinformation, and the second block being adjacent to the first block, thedecoder further configured to, determine a filter strength value inconsideration of a presence of the non-zero transform coefficientinformation, obtain quantization information related to at least one ofthe first and second blocks, compare a value related to the quantizationinformation with a constant value, and calculating values of the pixelson a block boundary between the first and second blocks, based on thefilter strength value and a result from the comparing, the calculatingincludes performing a bit shift operation, wherein the bit shiftoperation uses an integer, wherein the integer is greater than one, andthe calculating further includes, calculating values of at least twosuccessive pixels in the first block based on pixel values of the atleast two successive pixels in the first block, and calculating valuesof at least two successive pixels in the second block based on pixelvalues of the at least two successive pixels in the second block.